
<!DOCTYPE html>

<html lang="en">
  <head>
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="generator" content="Docutils 0.19: https://docutils.sourceforge.io/" />

    <title>Gold standard test jobs &#8212; LAVA 2024.05 documentation</title>
    <link rel="stylesheet" type="text/css" href="_static/pygments.css" />
    <link rel="stylesheet" type="text/css" href="_static/bootstrap-sphinx.css" />
    <script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script>
    <script src="_static/jquery.js"></script>
    <script src="_static/underscore.js"></script>
    <script src="_static/_sphinx_javascript_frameworks_compat.js"></script>
    <script src="_static/doctools.js"></script>
    <script src="_static/sphinx_highlight.js"></script>
    <link rel="shortcut icon" href="_static/favicon.ico"/>
    <link rel="index" title="Index" href="genindex.html" />
    <link rel="search" title="Search" href="search.html" />
    <link rel="next" title="Standard test job for QEMU - Stretch amd64" href="standard-qemu-stretch.html" />
    <link rel="prev" title="Providing gold standard images" href="gold-standards.html" />
    <link rel="canonical" href="https://docs.lavasoftware.org/lava/standard-test-jobs.html" />
  
<meta charset='utf-8'>
<meta http-equiv='X-UA-Compatible' content='IE=edge,chrome=1'>
<meta name='viewport' content='width=device-width, initial-scale=1.0, maximum-scale=1'>
<meta name="apple-mobile-web-app-capable" content="yes">
<script type="text/javascript" src="_static/js/jquery-1.12.4.min.js"></script>
<script type="text/javascript" src="_static/js/jquery-fix.js"></script>
<script type="text/javascript" src="_static/bootstrap-3.4.1/js/bootstrap.min.js"></script>
<script type="text/javascript" src="_static/bootstrap-sphinx.js"></script>


  </head><body>

  <div id="navbar" class="navbar navbar-default navbar-fixed-top">
    <div class="container">
      <div class="navbar-header">
        <!-- .btn-navbar is used as the toggle for collapsed navbar content -->
        <button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".nav-collapse">
          <span class="icon-bar"></span>
          <span class="icon-bar"></span>
          <span class="icon-bar"></span>
        </button>
        <a class="navbar-brand" href="index.html"><span><img src="_static/lava.png"></span>
          LAVA</a>
        <span class="navbar-text navbar-version pull-left"><b>2024.05</b></span>
      </div>

        <div class="collapse navbar-collapse nav-collapse">
          <ul class="nav navbar-nav">
            
                <li><a href="genindex.html">Index</a></li>
                <li><a href="contents.html">Contents</a></li>
            
            
              <li class="dropdown globaltoc-container">
  <a role="button"
     id="dLabelGlobalToc"
     data-toggle="dropdown"
     data-target="#"
     href="index.html">Site <b class="caret"></b></a>
  <ul class="dropdown-menu globaltoc"
      role="menu"
      aria-labelledby="dLabelGlobalToc"><ul class="current">
<li class="toctree-l1"><a class="reference internal" href="index.html">Introduction to LAVA</a></li>
<li class="toctree-l1 current"><a class="reference internal" href="contents.html">Contents</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="glossary.html">Glossary of terms</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="support.html">Getting support</a></li>
</ul>
</ul>
</li>
              
                <li class="dropdown">
  <a role="button"
     id="dLabelLocalToc"
     data-toggle="dropdown"
     data-target="#"
     href="#">Page <b class="caret"></b></a>
  <ul class="dropdown-menu localtoc"
      role="menu"
      aria-labelledby="dLabelLocalToc"><ul>
<li><a class="reference internal" href="#">Gold standard test jobs</a><ul>
<li><a class="reference internal" href="#metadata">Metadata</a></li>
</ul>
</li>
</ul>
</ul>
</li>
              
            
            
              
                
  <li>
    <a href="gold-standards.html" title="Previous Chapter: Providing gold standard images"><span class="glyphicon glyphicon-chevron-left visible-sm"></span><span class="hidden-sm hidden-tablet">&laquo; Providing gol...</span>
    </a>
  </li>
  <li>
    <a href="standard-qemu-stretch.html" title="Next Chapter: Standard test job for QEMU - Stretch amd64"><span class="glyphicon glyphicon-chevron-right visible-sm"></span><span class="hidden-sm hidden-tablet">Standard test... &raquo;</span>
    </a>
  </li>
              
            
            
            
            
              <li class="hidden-sm"></li>
            
          </ul>

          
            
<form class="navbar-form navbar-right" action="search.html" method="get">
 <div class="form-group">
  <input type="text" name="q" class="form-control" placeholder="Search" />
 </div>
  <input type="hidden" name="check_keywords" value="yes" />
  <input type="hidden" name="area" value="default" />
</form>
          
        </div>
    </div>
  </div>

<div class="container">
  <div class="row">
    <div class="body col-md-12 content" role="main">
      
  <section id="gold-standard-test-jobs">
<span id="using-gold-standard-files"></span><span id="index-0"></span><h1>Gold standard test jobs<a class="headerlink" href="#gold-standard-test-jobs" title="Permalink to this heading">¶</a></h1>
<p>The next step after the <a class="reference internal" href="explain_first_job.html#explain-first-job"><span class="std std-ref">first example job</span></a>,
is to move on to <em>gold standard</em> test jobs. These include a test job
for QEMU and test jobs for readily available ARM devices like
<em>cubietruck</em> and <em>beaglebone-black</em>.</p>
<div class="admonition seealso">
<p class="admonition-title">See also</p>
<p><a class="reference internal" href="gold-standards.html#creating-gold-standard-files"><span class="std std-ref">Creating a gold standard image</span></a></p>
</div>
<p>Test writers are strongly recommended to follow these examples to have a known
working setup for later test jobs. These standard test jobs are part of the
regular functional testing performed by the LAVA software team prior to each
release. If you have problems debugging a test job failure on any instance of
LAVA, your recommended first action is to submit the closest standard test job
and compare the results against other instances like
<code class="docutils literal notranslate"><span class="pre">staging.validation.linaro.org</span></code>. To make this easier, all test writers are
recommended to use version control for all test jobs and to keep a local copy
of the closest standard test job for quick reference.</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>A gold standard deployment for one device type is not necessarily
supported for a second device type. Some devices will never be able to
support all deployment methods due to hardware constraints or the lack of
kernel support. This is <strong>not</strong> a bug in LAVA. If a particular deployment is
supported but not stable on a device type, there will not be a gold standard
image for that deployment. Any issues in the images using such deployments
on that type are entirely down to the test writer to fix.</p>
</div>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>Device type templates and other instance-specific configuration could
affect the standard test jobs on specific instances. If a standard test job
fails, check with the admins for that particular instance. The standard
tests described here use support in the default device-type templates
shipped with LAVA and are routinely tested on
<code class="docutils literal notranslate"><span class="pre">staging.validation.linaro.org</span></code>.</p>
</div>
<section id="metadata">
<span id="job-metadata"></span><span id="index-1"></span><h2>Metadata<a class="headerlink" href="#metadata" title="Permalink to this heading">¶</a></h2>
<p>Standard test jobs include <a class="reference internal" href="glossary.html#term-metadata"><span class="xref std std-term">metadata</span></a> which links the test job back to
the build information, logs and scripts.</p>
<p>Metadata consists of a series of key value pairs where the key must be a single
word (no whitespace, underscores and hyphens are allowed). Keys must be unique
within the metadata.Keys used in metadata should be consistent across
particular groups of test jobs with data specific to that test job in the
value.</p>
<p>Ensure that you change the <strong>metadata</strong> to point at your local repository so
that you can easily distinguish between the results with and without your
modifications:</p>
<pre class="code yaml literal-block"><code><span class="name tag">metadata</span><span class="punctuation">:</span><span class="whitespace">
  </span><span class="name tag">source</span><span class="punctuation">:</span><span class="whitespace"> </span><span class="literal scalar plain">https://gitlab.com/lava/lava</span><span class="whitespace">
  </span><span class="name tag">path</span><span class="punctuation">:</span><span class="whitespace"> </span><span class="literal scalar plain">doc/v2/examples/test-jobs/qemu-amd64-standard-stretch.yaml</span></code></pre>
<p>Automated test jobs should always use the metadata to specify:</p>
<ul class="simple">
<li><p>the source repository of the job definition</p></li>
<li><p>the path to the job definition within the source repository</p></li>
<li><p>the details of the build</p></li>
<li><p>links to the build logs</p></li>
<li><p>links to the build scripts.</p></li>
</ul>
<p>These details allow other users to adapt and modify the build artifacts for
further tests.</p>
<p>The set of gold standard jobs has been defined in association with the Linaro
QA team. These jobs will provide a known baseline for test definition writers,
in a similar manner to the way the existing QA test definitions provide a base
for more elaborate testing.</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>Metadata also exists in the Lava Test Shell Definition but this is
only used when writing the test shell definition, it is not picked up by
LAVA. It remains useful to record the maintainer of a test shell definition,
the scope and the kinds of devices for which the definition is expected to
be useful. see also <a class="reference internal" href="first-job.html#test-definitions"><span class="std std-ref">Test Definitions</span></a>
To explicitly add limited metadata to the job from the Lava Test Shell
Definition, see <a class="reference internal" href="writing-tests.html#test-case-references"><span class="std std-ref">Recording test case references</span></a>.
The job will automatically contain metadata relating to the URL, commit
hash and path of the test shell definition(s) used in the test job.</p>
</div>
</section>
</section>


    </div>
      
  </div>
</div>
<footer class="footer">
  <div class="container">
    <p class="pull-right">
      <a href="#">Back to top</a>
      
    </p>
    <p>
        &copy; Copyright 2010-2019, Linaro Limited.<br/>
      Created using <a href="http://sphinx-doc.org/">Sphinx</a> 5.3.0.<br/>
    </p>
  </div>
</footer>
  </body>
</html>